fix: better handle errors in static analysis #271
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
giovanni-guidini
requested review from
dana-yaish,
scott-codecov and
matt-codecov
Codecov Report
@@ Coverage Diff @@
## master #271 +/- ##
==========================================
- Coverage 94.95% 94.89% -0.07%
==========================================
Files 78 78
Lines 2657 2684 +27
==========================================
+ Hits 2523 2547 +24
- Misses 134 137 +3
Flags with carried forward coverage won't be shown. Click here to find out more.
|
This was needed already.
But ultimately inspired by a user tha had the following error:
```
File "codecov_cli/services/staticanalysis/analyzers/python/node_wrappers.py", line 41, in do_visit
first_if_statement = first_if_statement.children[0]
IndexError: list index out of range
```
Despite my best efforts, I was only able to reproduce this error using invalid Python code
(i.e. an empty `if` statement)
Regardless of what caused this issue in the 1st place, there may be valid reasons
for someone to have malformed code in their repo (depending what the repo is 👀)
So we should be able to still upload _something_ and ignore the failures.
The tricky thing here is that without the static analysis data ATS can't work well.
And we can't be sure that the analyzer doesn't have bugs. I don't think this case in
particular is one of them, but I could be wrong.
So these changes actually are a way to surface processing errors AND ignore the files
that generated them, WHILE also surfacing errors to users. They are surfaced twice.
Ideally users would look at them and let us know if something is not right.
But maybe we should still fail the static analysis step in their CI? 🧐
giovanni-guidini
force-pushed
the
gio/fix-static-analysis-index-error
branch
from
22b4630 to
dfc45f5
Compare
scott-codecov
approved these changes
Sep 29, 2023
There was a problem hiding this comment.
Looks good to me.
Like we chatted about - it would make sense to me if we could catch this sort of malformed error a little higher up. i.e. when tree-sitter is parsing the source into an AST does it know that an if with no children is not valid Python?
Looks like tree-sitter-python uses the Python grammar here: https://docs.python.org/3/reference/grammar.html. Not sure how it would even parse an if statement without children since the grammar doesn't allow it.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This was needed already.
But ultimately inspired by a user tha had the following error:
Despite my best efforts, I was only able to reproduce this error using invalid Python code
(i.e. an empty
ifstatement)Regardless of what caused this issue in the 1st place, there may be valid reasons
for someone to have malformed code in their repo (depending what the repo is 👀)
So we should be able to still upload something and ignore the failures.
The tricky thing here is that without the static analysis data ATS can't work well.
And we can't be sure that the analyzer doesn't have bugs. I don't think this case in
particular is one of them, but I could be wrong.
So these changes actually are a way to surface processing errors AND ignore the files
that generated them, WHILE also surfacing errors to users. They are surfaced twice.
Ideally users would look at them and let us know if something is not right.
But maybe we should still fail the static analysis step in their CI? 🧐